IS THIS ANYTHING? (AND WILL IT FLOAT?)


Those wacky pollsters at the U.S. Census must looooove Cleveland. It used to be they'd put out one of their estimates, or projections, or annual updates, and hardly anyone would notice. It was, like, maybe the AP would stick something on the wire for local papers to run on page 14.

But now the Federal datameisters know there's one city that's waiting for their next press release with bated breath, panting to get its hands on their next ranking list, desperate to find out where it stands. Cleveland takes them seriously.

Last August the Plain Dealer read a press release about the 2003 American Community Survey and discovered poverty. There it was, in black and white -- we're the poorest big city! Never mind that the ACS contains some very questionable numbers in other categories. Never mind that all the ACS data comes with big lower-to-upper-bound ranges. Never mind that the basic "revelation" -- that Cleveland's income numbers put it at or near the urban bottom -- had stuck out like a sore thumb in the real Census report two years earlier (and in 1990 too, for that matter). The PD decided it now had some big news on its hands -- and so for the last year, we've all been talking about "Cleveland, the poorest city". (This year's ACS is due out in two months. Do you think the PD will call it progress if we go from 68th to 66th in household incomes?)

Now it seems that this year's Big Census Cleveland News Story is going to be "not just poorest, but smallest". The Census released its annual (2004) Population Estimates yesterday, and they showed Cleveland losing 22,000 people since 1999. Wowza! Clear the front page! Cleveland has its smallest population since 1900!

Since 22,000 lost residents is about 4,000 each year, compared to a ten-year loss average of 2,700 a year in the previous decade -- and we have a mayor running for re-election who promised to get the numbers moving in the opposite direction -- the high story interest might be legitimate... if the numbers were meaningful.

But a closer look at the Population Estimates raises serious doubts about that. As Letterman might say: "Is this anything, Paul? No, I think we'd have to say it's not anything" -- certainly not anything that deserves half the front page.

Here's a good place to start -- the Estimates for Cleveland for 1990, 1995 and 1999, still available in the archives:

1990 -- 505,450
1995 -- 501,228
1999 -- 501,662

And what was Cleveland's actual Census count in 1999, when the official 2000 Census was published? 478,403. That's 23,000 fewer people than the Census "estimated", using the same methodology they're still using. How far off do you have to be to be wrong?

But I'm not complaining about the data itself. I'm complaining about the way the PD used it.

See, from the placement, size and excitable attitude of yesterday's PD article, you'd naturally think that the Census had released the findings of a major new investigation. But that's not true at all. The Census doesn't go out and count heads again every year. No, the Population Estimate is called an Estimate for a very good reason: That's all it is. An informed guess. A ballpark idea. An approximation.

Here's the methodology page for the 2004 Estimate process. It's rough going, but give it a try. You'll notice that the Census didn't actually estimate (or survey, or count) city populations directly. Rather, they tried to figure out what might have happened to county populations based on birth and death records, tax returns, immigration data, etc. Then they tried to allocate the county totals among localities like Cleveland, primarily by way of housing units -- how many we started with, new building permits, demolitions, etc. Both stages of this process are fraught with difficulties, especially in a city like Cleveland with lots of hard-to-trace people and property.

You wouldn't expect a statistics-based updating process like this to get all that close to empirical reality. It's not a "junior Census" (unlike the ACS, which at least does significant polling to generate new data every year.) The Population Estimates are limited, interim planning tools that (no doubt) have legitimate uses in the hands of research professionals who understand their limitations.

But that doesn't seem to describe the editors of the Plain Dealer.

Incidentally, in addition to taking the Population Estimates waaay too seriously, the PD apparently doesn't understand that its big headline -- "Cleveland population lowest since 1900" -- has been true every year since at least 1990.

To sum up: Contrary to appearances, no, this is not anything. And to answer the other key Letterman question: No, I don't think it will float. Too many holes.